test: Strengthen assertions and isolate external HTTP calls - #985
marandaneto wants to merge 2 commits into
Conversation
Test coverage comparisonMeasured on Python 3.13.13 with the same branch-coverage command before and after. Coverage includes runtime code under
The six baseline failures involved external HTTP dependencies under the network guard. The repaired tests use controlled transport responses instead. The main improvement is stronger assertions and test isolation, not broader execution coverage. As one check, changing the tracing default flush interval to 50 seconds in memory passed the old defaults test but failed the repaired test. These numbers cover the root suite only. Separate validation passed for OpenFeature (68 tests), Django middleware and exception capture (7 tests), and the compliance adapter (23 tests). The comprehensive semantic audit remains incomplete. Live credential-dependent tests, MCP v2, and the full Python-version matrix were not run. |
posthog-python Compliance ReportDate: 2026-09-27T14:28:43.255747+00:00 ✅ All Tests Passed!121/121 tests passed Capture_V1 Tests✅ 95/95 tests passed View Details
Capture_Ai Tests✅ 5/5 tests passed View Details
Feature_Flags Tests✅ 17/17 tests passed View Details
Feature_Flags_Local_Evaluation Tests✅ 4/4 tests passed View Details
|
|
[Low risk] Test improvements and assertion strengthening across test files. The PR appears safe to merge, with non-blocking gaps in tests intended to protect timing and registry coverage. Reviews (1) · Last reviewed commit: "test: Strengthen assertions and isolate ..." |
💡 Motivation and Context
Several tests could pass without exercising the behavior they claimed to protect. Some also depended on external HTTP services, scheduler timing, or assertions raised inside worker callbacks that catch exceptions.
This test-only change replaces those HTTP calls with controlled responses and strengthens checks for retry counts, event payloads, exception propagation, truncation, copy isolation, and lifecycle behavior. Consumer tests now wait for delivery and stop their workers. A controlled-clock test checks the remaining flush interval. Registry key checks catch missing or untested OpenAI resources, and placeholder checks retain both explicit expectations and coverage of every registered entry. LangChain error tests still exercise the real callback lifecycle through a mocked HTTP transport.
These are the validated repairs from a partial test audit. This PR does not claim that every repository test has been reviewed. No production code or public API changes are included.
💚 How did you test it?
origin/mainreported no actionable findings for4544aa5.The full-suite and coverage results above were measured at
107b3fb. After review follow-ups at4544aa5, 53 focused consumer and OpenAI resource tests, the API checker tests, and Ruff passed. An early-flush mutation passed the previous timing test and failed the new controlled-clock test. Extra and missing wrapper keys were both rejected.Validation used Python 3.13.13. Live credential-dependent tests, MCP v2, and the full Python-version matrix were not run.
📝 Checklist
If releasing new changes
sampo addto generate a changeset file🤖 Agent context
Autonomy: Human-driven (agent-assisted)
Pi used read-only subagents for test inventories and review, then applied and validated focused test repairs in a dedicated worktree. Tools included Git, pytest, coverage.py, Ruff, mypy, and the autoreview helper. The audit remains incomplete, and unresolved review scope was not treated as approved. Local audit notes are kept outside the PR. No shared session link is available.